Improve Callbacks - #191

AbstractMCMC.jl documentation for PR #191 is available at:
https://TuringLang.github.io/AbstractMCMC.jl/previews/PR191/

Can I request that you merge this into a branch that is not main?

On the Turing.jl PR can you use https://pkgdocs.julialang.org/v1/toml-files/#The-%5Bsources%5D-section to point to the unreleased branch of AbstractMCMC?

I want to avoid a situation where we do stuff on Turing, realise that something here needs to be changed, and then you have to make a breaking release on AbstractMCMC to fix it.

Once the Turing code is fine then we can release a new minor version here.

I would go further and ask why there are extras and stats separately? There's a filter that allows the user to control which stats they want, right? Can't the sampler bundle literally everything into stats, and then the user can be responsible for choosing which ones they want?

We kept them separate based on @yebai's suggestion to distinguish between step-level statistics and "extras" (hyperparameters).

@penelopeysm, since this PR isn't breaking and other research work depends on it, are you happy if we merge it to main and release it immediately?

No not really, because I would rather not find out that there's something that needs to be changed in the interface, and then having to release a breaking version of AbstractMCMC.

other research work depends on it

Am I missing something, or is it just the Turing PR?

If it's just the Turing PR, as I said in my last comment somewhere up there, you can use [sources] in Project.toml to point to an unreleased branch.

This is what Markus and I have been doing for DPPL releases for quite a while now.

(If you're like, absolutely, 100%, certain that the interface is final and it will not need to be changed, then sure merge to main and release -- but I don't have that confidence myself.)

It's very helpful for some MCMC research work : )

Let's be slightly more flexible here, since AbstractMCMC.ParamsWithStats is likely to be overloaded only by Tuing.jl and JuliaBUGS.jl in the near future, and @shravanngoswamii can help keep them in sync.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.00%. Comparing base (31db894) to head (2793220).
⚠️ Report is 1 commits behind head on main.

@@     Coverage Diff     @@
##   main   #191   +/-   ##
===========================
===========================

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@shravanngoswamii can help keep them in sync.

Happy to iterate on it anytime!